1. Field
Embodiments described in this disclosure relate to preventing malicious payloads from exploiting vulnerabilities on a server. More specifically, embodiments provide attack-resistant verification of auto-generated anti-malware signatures.
2. Description of the Related Art
As the internet has grown, malware has become a major concern for businesses and individuals connected to the internet and other networks. Malware (i.e., worms, spyware, etc.) is frequently delivered using a payload, which may target a particular vulnerability of an application. For example, server applications are widely available to respond to messages from any requesting party (e.g., a web server configured to respond to HTTP requests) and network messages may be crafted with payloads intentionally designed to exploit a vulnerability of a server. For example, a network message may include a payload that causes a buffer overflow on a vulnerable system, allowing a remote attacker to execute arbitrary shell code on a host system. In some cases, a payload designed to exploit a vulnerability may be identified using a signature (e.g., a hash value computed over all or a portion of the payload or by identifying specific arguments or combinations of arguments). If a particular payload signature is identified as being characteristic of a malicious payload, then messages carrying that payload may be blocked.
Because of the large number and variety of malware attacks that occur today, security systems, known as intrusion prevention systems (IPS) have been developed to automatically block malicious traffic using a database of malicious payload signatures. For example, a network administrator may deploy an IPS on a host system or at a gateway edge between a local network and the internet. When the IPS detects a malicious payload signature in a payload addressed to a server on that host, it may simply drop that message. That is, the IPS intercepts the message, preventing it from being forwarded towards a destination. Further, the IPS raises an alarm and even block future traffic from the source address. Of course, to be effective, an IPS needs to correctly distinguish between malicious and non-malicious payloads, and the signatures used by the IPS are a significant factor in the performance of an IPS. Typically, an IPS signature database may be updated periodically by security experts who study network traces after a worm has been released. Additionally, an IPS may employ an automatic signature generation security system to help maintain an effective signature database. For example, the IPS may monitor network traffic to identify a new attack vectors (i.e., new payloads) and produce a signature using pattern-based analysis. For example, for polymorphic malware, longest common token subsequence (LCTS) signatures are commonly used. Furthermore, malicious payload signatures may be grouped according to the vulnerability that is targeted by each payload.